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DETAILED ACTION 

1 . This communication is responsive to tine application filed 6/2/2009. 

2. Claims 1-23 are pending in this application. Claims 1,12 and 23 are independent 
claims. This action is made Non-Final. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the phor art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1-23 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Dempski (US 2006/0244677) in view of Foxlin (US 2002/0024675). 

Regarding claim 1 , Dempski discloses a system that enables a user to view a 
virtual control panel (see paragraph [0013] where it is discussed that selected data 
associated with a device is displayed on a wearable display). 

Dempski does not expressly disclose user interaction with the virtual control 
panel using a user controlled pointing object. However, Foxlin discloses a head- 
mounted virtual reality system capable of tracking a hand-mounted object relative to a 
user's head (see paragraph [0057] and [0058]). It would have been obvious to an 

artisan at the time of the Invention to include Foxlin's teachings in Dempskl's user 
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interface in an effort to enable user to intuitively interact with applications in a virtual 
environment (see paragraphs [0004] and [0076]). 

The modified DempskI discloses a first tracking unit (see Foxlln, fig 1 , 30) 
adapted to capture data representing the position of the pointing object (see Foxlin, fig 

1,14), 

a portable Identification element (see Foxlin, fig 1, 10), 

a second tracking unit adapted to capture data representing the position of the 
identification element (see Foxlin, fig 1, 30), 

a storage unit (see Dempski, fig 1 , 36, 38, 40, 42) storing at least one pre-defined 
graphical interface representing a control panel of a device, said graphical interface 
comprising an interface for user interactions with the device (see DempskI, paragraph 
[0012] "The databases may also store data about the work environment, object details, 
current status and location of workers, etc."), 

a graphics unit generating a graphical representation of the control panel based 
on said stored graphical interface (see Dempski, paragraph [0013] "selecting data from 
a memory storage with the data being associated with one of the objects associated 
with one of the visual markers, and then displaying the data on a wearable display worn 
by the operator."), 

a registering unit registering said graphical representation of the control panel in 
a fixed relation to said portable identification element, based on said data representing 
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the position of tine identification element, to produce a virtual control panel (see Foxlin, 
paragraphs [0090], [0091] and [0096]), 

a display unit showing the user a view comprising the real world and the virtual 
control panel projected in a fixed relation to said portable identification element (see 
Dempski, paragraph [0015] where the see-through display is discussed), and 

an application unit performing actions in response to the users interactions with 
the virtual control panel, and determining which actions to be performed based on the 
position of said user controlled pointing object in relation to the identification element 
(see Foxlin, paragraph [0105] where selecting and manipulating objects is discussed). 

Regarding claim 2, Foxlin discloses the system is adapted to modify the 
appearance of the virtual control panel in response to interactions between the user 
controlled pointing object and the virtual control panel (see paragraph [0105] where 
selecting and manipulating objects is discussed). 

Regarding claim 3, Dempski discloses said graphical interface is adapted to 
display data from the device and wherein the system is adapted to generate a graphical 
representation of the data and to display the data on the virtual control panel (see 
paragraph [0013] "selecting data from a memory storage with the data being associated 
with one of the objects associated with one of the visual markers, and then displaying 
the data on a wearable display worn by the operator."). 
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Regarding claim 4, Foxlin discloses said user controlled pointing object is a 
handheld pointing device or a part of the user's body (see paragraphs [0007] and 
[0008]). 

Regarding claim 5, Dempski discloses the storage unit is adapted to store a 
plurality of graphical interfaces, each representing a control panel of a particular device, 
wherein the system is adapted to generate and display a plurality of graphical 
representations of control panels for different devices based on said stored graphical 
interfaces of the devices (see paragraph [0013] "selecting data from a memory storage 
with the data being associated with one of the objects associated with one of the visual 
markers, and then displaying the data on a wearable display worn by the operator."), 
and wherein the system further comprises: means for identifying which of the stored 
control panels to be displayed (see paragraph [0016] where the unique identifiers 
associated with the markers is discussed). 

Regarding claim 6, Dempski discloses said means for identifying which of the 
stored control panels to be displayed comprises a recognition unit for recognizing and 
identifying devices in the environment of the user, and wherein the system is adapted to 
determine which of the stored control panels to be displayed based on which of the 
devices is identified (see paragraph [0016] where the unique identifiers associated with 
the markers is discussed). 
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Regarding claim 7, Dempsl^i discloses said recognition unit is adapted to 
recognize and identify unique identification markings on the devices (see paragraph 
[0016] where the unique identifiers associated with the markers is discussed). 

Regarding claim 8, Dempski discloses the system is arranged so that it changes 
the virtual control panel displayed when another device is recognized and identified, and 
when the user has accepted the device (see paragraph [0013] "selecting data from a 
memory storage with the data being associated with one of the objects associated with 
one of the visual markers, and then displaying the data on a wearable display worn by 
the operator."). 

Regarding claim 9, Foxlin discloses said portable identification element is 
adapted to be carried by the user during interaction with the virtual control panel (see 
paragraph [0058] where the wearable computer, 10 is discussed). 

Regarding claim 10, Foxlin discloses said portable identification element is 
attachable to the body of the user (see paragraph [0058] where the wearable computer, 
10 is discussed). 

Regarding claim 1 1 , Dempski discloses said display unit comprises a wearable 
display device showing the user said view (see paragraph [0008] where the wearable 
see-through display, 12 is discussed). 

Claims 12 and 13 are similar in scope to claims 1 and 2, respectively, and are 
therefore rejected under similar rationale. 
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Regarding claim 14, Foxlin discloses defining a two-way communication between 
the virtual control panel and the device, sending information to the device regarding the 
users actions with the virtual control panel, receiving data from the device, generating a 
graphical representation of the received data and displaying the data on the virtual 
control panel (see paragraph [0105] where selecting and manipulating objects is 
discussed). 

Regarding claim 15, Foxlin discloses said data is displayed on the virtual control 
panel in response to interactions between the user controlled pointing object and the 
virtual control panel (see paragraph [0057] and [0058] where the hand-mounted beacon 
14 is discussed). 

Regarding claim 16, Dempski discloses storing a plurality of pre-defined 
graphical interfaces, each representing a control panel of a particular device, 
determining which of the stored control panels to be displayed, and generating a 
graphical representation of the control panel to be displayed based on the pre-defined 
graphical interface of the control panel to be displayed (see Dempski, paragraph [0012] 
"The databases may also store data about the work environment, object details, current 
status and location of workers, etc."). 

Regarding claim 17, Dempski discloses at least one of the stored graphical 
interfaces comprises more than one graphical view to be displayed on the virtual control 
panel, and which of the views to be displayed is determined based upon the users 
actions (see paragraph [0022] "The longer that the operator gazes upon the object the 
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system may optionally be programmed to provide additional information regarding the 
object."). 

Regarding claim 18, Dempski discloses recognizing and identifying a device, 
determining which of the stored control panels to be displayed based on the identified 
device, and generating graphical representation of the control panel of the identified 
device based on the stored graphical interface of the identified device and displaying a 
view comprising the real world and the virtual control panel of the identified device 
projected in a fixed relation to said portable identification element (see paragraph [0013] 
"selecting data from a memory storage with the data being associated with one of the 
objects associated with one of the visual markers, and then displaying the data on a 
wearable display worn by the operator."). 

Regarding claim 19, Dempski discloses each device is provided with a unique 
identification marking and a device is recognized by identifying its unique identification 
marking (see paragraph [0016] where the unique identifiers associated with the markers 
is discussed). 

Regarding claim 20, Dempski discloses the virtual control panel displayed is 
changed when another device is recognized and identified, and when the user has 
accepted the device (see paragraph [0013] "selecting data from a memory storage with 
the data being associated with one of the objects associated with one of the visual 
markers, and then displaying the data on a wearable display worn by the operator."). 
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Regarding claim 21, Foxlin discloses said portable identification element is 
carried by the user during interaction with the virtual control panel (see paragraph 
[0058] where the wearable computer, 10 is discussed). 

Regarding claim 22, Dempski discloses the virtual control panel comprises virtual 
interaction members and an audio and/or visual feedback is generated when the user 
activates any of the virtual interaction members (see paragraph [0022] "the system may 
be responsive to voice command of the operator"). 

Claim 23 is similar in scope to claim 1 and is therefore rejected under similar 
rationale. 



Conclusion 

5. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to RASHAWN TILLERY whose telephone number is 571- 
272-6480. The examiner can normally be reached on M-F 8:30 AM - 4:30 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Dennis Chow can be reached on 571-272-4124. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding tine status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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